| # of distinct signatures | # of sentences | Ratio |
|---|---|---|
| 29992 | 30000 | 0.9997 |
| Hash value | Count |
|---|---|
| 731570722 | 2 |
| 768749748 | 2 |
| 288740083 | 2 |
| 986295512 | 2 |
| 511313655 | 2 |
| 411833945 | 2 |
| 138888397 | 2 |
| 1107367029 | 2 |
| 1868261280 | 1 |
| 2094382603 | 1 |
| 1624236581 | 1 |
| 1460006210 | 1 |
| 124850389 | 1 |
| 1335229636 | 1 |
| 1593952557 | 1 |
| 1847647742 | 1 |
| 1369222554 | 1 |
| 92464101 | 1 |
| 1448536857 | 1 |
| 789810672 | 1 |
Again, sentence similarity is considered. The approach is in parallel to 4.7.1.1, but we use a hash value for the sentence instead of the sentence signature.
First we count the different hash values compared to the number of sentences, then we list the hash values with the highest multiplicities.
First table:
select aa.a,bb.b, aa.a/bb.b from (select count(distinct hash) as a from para_s) aa,(select count(*) as b from para_s) bb;
Second table:
select hash, count(*) as anz from para_s group by hash order by anz desc limit 20;
4.7.3.2 Sentences with Most Frequent Hash Values